In general, a server may be configured to provide information to one or more clients according to a client/server model of information delivery. In this model, the server is a storage system that typically contains one or more mass storage devices, such as magnetic hard disks, in which information may be stored and retrieved as desired. The server is usually deployed over a computer network comprising a geographically distributed collection of interconnected communication links, such as Ethernet, optical or wireless links, that allow the clients to remotely access the server's stored information. The clients may include network devices, such as computers, that are directly or indirectly attached to the server, e.g., via point-to-point links, shared local area networks (LAN), wide area networks (WAN) or virtual private networks (VPN) implemented over a public network such as the Internet. Yet other clients may include software applications executing on computers that are configured to communicate with the server.
In some client/server arrangements, the server may be configured to operate as a network cache that buffers previously-accessed or frequently-accessed client information. As such, the network cache provides a set of clients with faster access to the buffered information than if they were to access the same information directly from a set of origin servers. For instance, the clients may be physically situated closer to the network cache than to the origin servers, or the clients may be able to access the cache over a lower latency (or higher bandwidth) data path, etc. The network cache's buffered information is typically in the form of an object store containing one or more predefined data objects which are made accessible to the clients. As used herein, a data object is any collection of data that is identifiable by a common name. For example, data objects may include conventional files, such as HyperText Mark-up Language (HTML) files (“web pages”), as well as other types of data objects known in the art.
Clients typically communicate with the network cache by exchanging discrete packets of data formatted according to predefined file-access protocols, such as the HyperText Transfer Protocol (HTTP), Network File System (NFS) protocol, Common Internet File System (CIFS) protocol, File Transfer Protocol (FTP), etc. A client may issue a file-access request that specifies, among other things, a specific data object to access and a particular operation to perform. The network cache receives the client request, processes the request, and when appropriate returns a response. For example, the client may issue a “read” request to retrieve a particular data object from the network cache and, in response, the cache may return a file-access response containing the client's requested data object.
It is often desirable for the network cache to scan client-requested data objects for viruses, spyware or other unauthorized content before the objects may be returned to their requesting clients. In the event that unauthorized content, such as a virus, is located in a client-requested data object, the client may be notified that the object is not currently accessible. Alternatively, the requested data object may be transformed, or “cleaned,” in order to remove the unauthorized content before the object is returned to the client. The network cache also may be configured to perform one or more content-filtering operations on the client's file-access request, e.g., to determine whether the client is authorized to access its requested data object.
The Internet Content Adaptation Protocol (ICAP) provides a mechanism for selectively transforming file-access requests and/or responses. The ICAP protocol employs a standard client/server arrangement in which an ICAP client communicates with an ICAP server that is configured to perform a set of ICAP services. For example, the ICAP services may include, inter alia, virus scanning, spyware control and content-filtering services. For each file-access request or response that the ICAP server processes, the server returns either a modified (“transformed”) or unmodified version of the request or response to the ICAP client, depending on which ICAP services were performed. The ICAP protocol may be used to perform various types of object-based, content-vectoring services and is generally described in more detail in RFC 3507 entitled Internet Content Adaptation Protocol (ICAP), by J. Elson et al., published April 2003, which is hereby incorporated by reference as though fully set forth herein.
According to the standard ICAP protocol (RFC 3507), the ICAP server is configured to store an ICAP Service Tag (“ISTag”) value that identifies the current software version or particular configuration of its ICAP services. The ISTag is an exemplary configuration identifier used to identify a current configuration. The configuration ID, e.g., the ISTag value, is included in every ICAP message sent from the ICAP server to the ICAP client. Thus, when the ICAP server modifies its ICAP services or otherwise changes its configuration, the server can communicate this change to the ICAP client by sending a new ISTag value in the next ICAP message forwarded to the client. In this manner, the ICAP client remains apprised of which version(s) of the ICAP services are being executed at the ICAP server.
The most common types of ICAP services performed by an ICAP server include content-filtering, virus scanning and spyware scanning. Each of these services typically relies on a corresponding database. For instance, the content-filtering services may maintain a database containing a list of authorized uniform resource identifiers (URI) or uniform resource locators (URL), the anti-virus services may maintain a database of known virus-signature patterns, and so forth. These conventional ICAP services often require frequent database updates to ensure that their databases contain up-to-date information. To communicate the database updates to the ICAP client, the ICAP server typically allocates a new ISTag value every time one of its databases is updated. For instance, the ISTag value reported by the ICAP server may be incremented, e.g., by one, after every database update.
A network cache typically executes an ICAP client that sends file-access requests and/or responses to a remote ICAP server. In conventional implementations, the network cache uses the ICAP server's ISTag value to determine which cached data objects need to be re-processed by the ICAP server. More specifically, the network cache typically maintains a hash table which is used to locate valid data objects in the network cache. In this context, a valid data object is a data object that has been processed by the most-recent version of the ICAP server's ICAP services. For instance, suppose that a data object stored in the network cache was last processed by the ICAP server when the server's ISTag value equaled one. In this case, the data object is a valid data object so long as the server's ISTag value remains equal to one. Otherwise, if the ICAP server updates its ICAP services and thus reports an updated ISTag value, e.g., equal to two, then the cached data object becomes invalid.
In practice, the network cache generates a hash key to locate a reference to a valid client-requested data object in the hash table. The hash key is generated by first combining the URL of the requested data object with the most-recent ISTag value reported by the ICAP server, and then hashing the combined URL and ISTag values, e.g., using a conventional hash function such as the Message Digest version 5 (MD5) function. The resultant hash key is used to index a matching hash-table entry corresponding to the client-requested data object. As is known in the art, the hash function is preferably selected to minimize the number of collisions in the hash table. However, in the event of a collision (i.e., two different URL and ISTag combinations generating the same hash key), the matching hash-table entry may be configured to reference multiple data objects, e.g., which may be logically organized as a linked list. When the client-requested data object cannot be located in the hash table, the data object is fetched from an appropriate origin server, sent to the ICAP server for processing and then stored on the network cache at a location that is referenced by the hash-table entry which is indexed using the hash key derived from the object's URL and the ICAP server's current ISTag value.
When the ICAP server updates its software version or configuration, and thus sends a new ISTag value to the network cache, the network cache uses the server's updated ISTag value to generate hash keys. Accordingly, if the network cache receives a client request for a data object that was previously cached using the ICAP server's old ISTag value, the requested data object will not be located in the hash table when a new hash key is derived for the object using the server's updated ISTag value. As a result, the unlocated client-requested data object has to be re-retrieved from an appropriate origin server, re-processed by the ICAP server, and stored again on the network cache at a location referenced in the hash table using the object's newly-generated hash key. The reference to the previously-cached version of the data object, which is no longer a valid data object, is eventually removed from the hash table by a selected aging mechanism. Specifically, the aging mechanism monitors the contents of the hash table and periodically removes (“purges”) hash-table entries that have not been accessed for a predetermined period of time.
Because the conventional network cache generates hash keys using the ICAP server's ISTag value, the network cache necessarily invalidates all of its cached data objects in response to receiving an updated ISTag value from the ICAP server. In other words, references to data objects that were stored in the hash table using the ICAP server's old ISTag value cannot be located by hash keys generated using the server's updated ISTag value. In this manner, the previously-cached data objects are invalidated since they cannot be located in the hash table, and these invalidated object references are eventually purged from the hash table after they have aged. While this conventional solution generally works well for ensuring that only valid data objects are locatable in the network cache's hash table, this conventional approach suffers various disadvantages.
Most notably, the process of invalidating every data object in response to an updated ISTag value significantly reduces the cache-hit ratio at the network cache and, consequently, increases the response time and network-bandwidth consumption required to process client requests. For instance, most system administrators configure their ICAP servers to periodically update their ICAP-service databases, e.g., around once every 1-2 hours. This frequent database updating causes the ICAP server to repeatedly update its ISTag value, which in turn prompts the network cache to frequently invalidate all of its cached data objects. As a result, the network cache experiences a significant reduction in its cache-hit ratio, i.e., the ratio of the number of valid client-requested data objects that can be located in the network cache relative to the total number of client-requested data objects searched for at the cache. The low cache-hit ratio usually experienced in an ICAP-configured network cache requires the cache to consume a substantial amount of network bandwidth re-retrieving invalidated data objects from the origin servers. In addition, the requesting clients often experience excessive time delays waiting for their requested data objects to be re-retrieved from the origin servers and then re-processed by the ICAP server before the objects can be returned from the network cache.
It has also been observed that, in many cases, client-requested data objects include object types that are not modified by the ICAP services. Thus, it is unnecessary to invalidate and re-retrieve these types of data objects in response to frequent changes in the ICAP server's ISTag value. For example, suppose that the ICAP server is configured to perform anti-virus scanning. In this scenario, it is unnecessary for the network cache to repeatedly invalidate and re-retrieve image, media and text files that are incapable of carrying software viruses, even if the ICAP server's ISTag value is frequently changing in response to virus-signature pattern updates. Similarly, for ICAP services like virus and spyware scanning, where the only possible object modification is to remove a software virus from a data object, it is unnecessary to invalidate and re-retrieve a “cleaned” version of a data object every time the ICAP server's ISTag value changes.
The foregoing disadvantages of the conventional network-cache implementation pose an object-cacheability problem for which there is currently no known solution except for disabling ICAP processing at the network cache. Since ICAP is deployed for critical tasks, such as anti-virus and spyware control, it is undesirable for a system administrator to disable it. It is therefore generally desirable to improve object cacheability in an ICAP-configured network cache so as to increase the network cache's cache-hit ratio, thereby allowing the cache to consume less network bandwidth and reduce its average latency for responding to client requests.